class: center, middle, inverse, title-slide # Commercial Bank Customer Retention Prediction ## APSTA-GE 2401: Statistical Consulting Seminar, Mid-Presentaion ### Tong Jin, Andy Tan, Zixuan Zhou ### New York University ### 10/26/2020 --- ## Topics - Introduction - Overview - Tasks - Details - Project Plan --- class: inverse, center, middle # Introduction --- ## Overview **Commercial Bank Customer Retention Prediction** - Active competition on [Data Castle](https://www.dcjingsai.com/v2/cmptDetail.html?id=439) - Initial: 10/26 - 1st Round: 10/26 - 12/10 - 2nd Round: 12/11 - 12/25 - Final Round: 12/26 - 1/10/2021 - Using the following machine learning techniques to predict whether a customer will **retent** (or **churn**): - Exploratory Data Analysis (EDA) - Data Mining - Supervised Learning --- ## Tasks Banks need a better understanding of customer demands as they expanding their business. Specifically, banks are interested in learning about customer's churn rate and the change of their financial status. The purpose of predicting customer's retention probability is for precision marketing in order to prevent customers from churning. 1. Build a model to predict retention probability based on real-world data records. 2. Present a business solution for precision marketing based on model results. --- ## Details - Data - Linked data tables in `.csv` format - `cust_no`: ID - Two datasets: - Train - `x_train`: features - `y_train`: label - Test - `x_test`: additional data records with label removed --- ## Details - Features There are 5 feature categories marked with different initials: - `X`: end-of-month balance, `ncol_X = 8` - structural, checking, savings, CDs, funds, loans, ... - `B`: customer behavior, `ncol_B = 7` - transfer (mobile, branch), transaction data, frequency, ... - `E`: big events, `ncol_E = 18` - new account opening, mobile app activation, large transfer, ... - `C`: savings, `ncol_C = 2` - number of saving products and amounts - `I`: customer information, `ncol_I = 20` - age, sex, level, occupation, income, indicators (marital, mobile app, mobile pay) ... **Total number of features: 55** --- ## Details - Dimensions **Train:** Number of Rows: ``` ## [1] 465441 ``` **Train validation:** ``` ## [1] 145296 ``` **Test:** Number of Rows: ``` ## [1] 76722 ``` Number of Columns: 1 (label) --- ## Details - Train Churn Rate <div id="htmlwidget-769805a65ae74a9af93b" style="width:576px;height:432px;" class="plotly html-widget"></div> <script type="application/json" data-for="htmlwidget-769805a65ae74a9af93b">{"x":{"data":[{"orientation":"v","width":[1,1,1],"base":[0,0,0],"x":[-1,0,1],"y":[22241,30237,92818],"text":["count: 22241<br />label: -1","count: 30237<br />label: 0","count: 92818<br />label: 1"],"type":"bar","marker":{"autocolorscale":false,"color":"rgba(89,89,89,1)","line":{"width":1.88976377952756,"color":"transparent"}},"showlegend":false,"xaxis":"x","yaxis":"y","hoverinfo":"text","frame":null}],"layout":{"margin":{"t":25.7412480974125,"r":7.30593607305936,"b":39.6955859969559,"l":54.7945205479452},"font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187},"xaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[-1.65,1.65],"tickmode":"array","ticktext":["-1","0","1"],"tickvals":[-1,0,1],"categoryorder":"array","categoryarray":["-1","0","1"],"nticks":null,"ticks":"","tickcolor":null,"ticklen":3.65296803652968,"tickwidth":0,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"y","title":{"text":"label","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"yaxis":{"domain":[0,1],"automargin":true,"type":"linear","autorange":false,"range":[-4640.9,97458.9],"tickmode":"array","ticktext":["0","25000","50000","75000"],"tickvals":[0,25000,50000,75000],"categoryorder":"array","categoryarray":["0","25000","50000","75000"],"nticks":null,"ticks":"","tickcolor":null,"ticklen":3.65296803652968,"tickwidth":0,"showticklabels":true,"tickfont":{"color":"rgba(77,77,77,1)","family":"","size":11.689497716895},"tickangle":-0,"showline":false,"linecolor":null,"linewidth":0,"showgrid":true,"gridcolor":"rgba(235,235,235,1)","gridwidth":0.66417600664176,"zeroline":false,"anchor":"x","title":{"text":"count","font":{"color":"rgba(0,0,0,1)","family":"","size":14.6118721461187}},"hoverformat":".2f"},"shapes":[{"type":"rect","fillcolor":null,"line":{"color":null,"width":0,"linetype":[]},"yref":"paper","xref":"paper","x0":0,"x1":1,"y0":0,"y1":1}],"showlegend":false,"legend":{"bgcolor":null,"bordercolor":null,"borderwidth":0,"font":{"color":"rgba(0,0,0,1)","family":"","size":11.689497716895}},"hovermode":"closest","barmode":"relative"},"config":{"doubleClick":"reset","showSendToCloud":false},"source":"A","attrs":{"98ef7c1d50ed":{"x":{},"type":"bar"}},"cur_data":"98ef7c1d50ed","visdat":{"98ef7c1d50ed":["function (y) ","x"]},"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1},"debounce":0},"shinyEvents":["plotly_hover","plotly_click","plotly_selected","plotly_relayout","plotly_brushed","plotly_brushing","plotly_clickannotation","plotly_doubleclick","plotly_deselect","plotly_afterplot","plotly_sunburstclick"],"base_url":"https://plot.ly"},"evals":[],"jsHooks":[]}</script> --- ## Grading **Cohen's Kappa Statistic** `$$\kappa = \frac{Pr(a) - Pr(e)}{1 - Pr(e)}$$` where `Pr(a)` is the predicted label, `Pr(e)` is the expected label value. - `< 0`: No agreement - `0 - 0.2`: slight - `0.21 - 0.4`: fair - `0.41 - 0.6`: moderate - `0.61 - 0.8`: substantial - `0.81 - 1`: perfect --- class: inverse, center, middle # Project Plan --- ## Project Plan - Programming Language: `Python` - Structure: - Data processing: - Cleaning - EDA - Feature Engineering (SVD) - Baseline Model: Logistic regression with elastic net - Test model: - Random forest - Gradient boosting machine - Multilayer perceptron - AUC for selection - Feature importance --- class: inverse, center, middle # Thank you!